Experimental Evaluation of Subject Matter Expert-oriented Knowledge Base Authoring Tools
نویسندگان
چکیده
We describe a large-scale experiment in which non-artificial intelligence subject matter experts (SMEs)—with neither artificial intelligence background nor extensive training in the task—author knowledge bases (KBs) following a challenge problem specification with a strong question-answering component. As a reference for comparison, professional knowledge engineers (KEs) author KBs following the same specification. This paper concentrates on the design of the experiment and its results—the evaluation of SMEand KE-authored KBs and SME-oriented
منابع مشابه
Evaluation of Mixed-Initiative Knowledge Base Development Methods and Tools
This paper presents a general framework for the empirical evaluation of mixed-initiative knowledge base development methods and tools. It also illustrates it with a knowledge acquisition experiment performed to validate a mixedinitiative method for acquiring problem solving rules directly from a subject matter expert. The main feature of this framework is that the evaluation is an integral part...
متن کاملParallel Knowledge Base Development by Subject Matter Experts
This paper presents an experiment of parallel knowledge base development by subject matter experts, performed as part of the DARPA’s Rapid Knowledge Formation Program. It introduces the Disciple-RKF development environment used in this experiment and proposes design guidelines for systems that support authoring of problem solving knowledge by subject matter experts. Finally, it compares Discipl...
متن کاملConstraint Authoring System: An Empirical Evaluation
Evaluation is an integral part of research that provides a true measure of effectiveness. This paper presents a study conducted to evaluate the effectiveness of CAS, a knowledge acquisition authoring system developed for generating the domain knowledge required for constraint-based tutoring systems with the assistance of a domain expert. The study involved a group of novice ITS authors composin...
متن کاملLessons Learned from Authoring for Inquiry Learning: A Tale of Authoring Tool Evolution
We present an argument for ongoing and deep participation by subject matter experts (SMEs, i.e. teachers and domain experts) in advanced learning environment (LE) development, and thus for the need for highly usable authoring tools. We also argue for the "user participatory design" of involving SMEs in creating the authoring tools themselves. We describe our experience building authoring tools ...
متن کاملTelling Aardvarks from Zebras with an Expert System Written in SAS®
Expert systems are a branch of artificial intelligence that encapsulate the knowledge and behaviors of a subject matter expert in their domain of expertise. The expert’s automated services then become available throughout the enterprise. Classification and diagnosis represent an important class of problems that have traditionally been a major objective of expert systems. An object-oriented expe...
متن کامل